Skip to content

ci: make deps-locked check resilient to upstream releases#65

Merged
ernestprovo23 merged 1 commit into
mainfrom
ci/fix-deps-locked-flaky
Jun 13, 2026
Merged

ci: make deps-locked check resilient to upstream releases#65
ernestprovo23 merged 1 commit into
mainfrom
ci/fix-deps-locked-flaky

Conversation

@ernestprovo23

Copy link
Copy Markdown
Member

Root cause

The "Verify lockfile is in sync with pyproject.toml" step in .github/workflows/deps-locked.yml ran uv pip compile with no baseline output file, which means uv resolves every dependency to the latest satisfying release on each run. The result is then diffed against the committed requirements-dev.lock. Any upstream package release that satisfies a pyproject constraint causes a non-empty diff and a RED check with zero repo change.

Concrete example: requirements-dev.lock pins pytest==9.0.3 (satisfies pytest>=9.0.3). pytest 9.1.0 was published upstream. The baseline-less recompile resolves to 9.1.0, the diff is non-empty, and the check fails on every PR and on main — despite no code change in this repo.

What changed

A single cp is prepended to the check step:

cp requirements-dev.lock /tmp/requirements-dev.regenerated.lock
uv pip compile --universal --generate-hashes --python-version 3.11 \
  --extra dev pyproject.toml -o /tmp/requirements-dev.regenerated.lock

Per uv 0.6.16 docs: "If the output file already exists, the existing versions will be preferred when resolving dependencies, unless --upgrade is also specified." Seeding the temp file from the committed lock means a pin that already satisfies the pyproject constraint (e.g. pytest==9.0.3 vs >=9.0.3) is preserved. The diff against the committed lock is empty and the check passes.

Why it is still durable

The real failure mode — someone bumps a pyproject dependency without regenerating the lock — still causes the recompile to diverge from the committed lock (the new/changed dep produces a different resolution even with the existing-pin preference), so the diff is non-empty and the check fails as intended. This was verified locally with a throwaway pyproject modification adding requests>=2.0.0 to the dev extras.

What was NOT touched

  • Step 1 (pip install --require-hashes -r requirements-dev.lock) is unchanged — this is the real supply-chain guarantee and remains the primary defense.
  • Step 3 (import check) is unchanged.
  • requirements-dev.lock is unchanged. pytest==9.0.3 remains the committed pin. No deps were bumped or regenerated.
  • All uses: entries remain SHA-pinned with version comments; test_workflow_pins.py passes.

Verification

Local reproduction against uv 0.6.16 on the committed lock (pytest 9.0.3 pinned, pytest 9.1.0 available upstream):

cp requirements-dev.lock /tmp/requirements-dev.check.lock
uv pip compile --universal --generate-hashes --python-version 3.11 \
  --extra dev pyproject.toml -o /tmp/requirements-dev.check.lock
diff <(grep -v '^#' requirements-dev.lock) <(grep -v '^#' /tmp/requirements-dev.check.lock)
# → empty diff, pytest==9.0.3 preserved in output

Full test suite: 511 passed, 1 skipped (uv run --extra dev python -m pytest -q in project .venv).

…only on pyproject change)

The "Verify lockfile is in sync" step previously ran uv pip compile with no
output-file baseline, which floats every dependency to the latest satisfying
release and then diffs against the committed lock.  Any upstream publish of a
new satisfying release (e.g. pytest 9.1.0 satisfying >=9.0.3 while the lock
pins 9.0.3) produces a non-empty diff and fails the job with zero repo change —
making the check permanently flaky on every open PR.

Fix: seed the temp output file from the committed lock before running uv pip
compile (cp requirements-dev.lock /tmp/requirements-dev.regenerated.lock, then
-o /tmp/requirements-dev.regenerated.lock).  Per uv 0.6.16 docs: "if the
output file already exists, the existing versions will be preferred … unless
--upgrade is specified."  This preserves pytest==9.0.3 (and all other pins)
when they already satisfy the pyproject constraint, so the diff is empty and
the check passes.  The real failure mode — pyproject adds/changes a dep without
the lock being regenerated — still produces a non-empty diff and fails.

Step 1 (pip install --require-hashes) and Step 3 (import check) are unchanged.
The committed requirements-dev.lock is unchanged; no deps were bumped.
@ernestprovo23 ernestprovo23 merged commit 8cc1fc3 into main Jun 13, 2026
9 checks passed
@ernestprovo23 ernestprovo23 deleted the ci/fix-deps-locked-flaky branch June 13, 2026 19:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant